'  1 


NPRDC-TR-91-1 


October  1990 


Long  Term  Memory  for  Knowledge  Taught  in  School 


George  B.  Semb 
John  A.  Ellis 
William  E.  Montague 


Reviewed  by 
Nick  Van  Mate 


Approved  by 
W.  H.  Wulfeck  II 


Released  by 
B.  E.  Bacon 
Captain,  U.  S.  Navy 
Commanding  Officer 
and 

Richard  C.  Sorenson 
Technical  Director,  Acting 


Approved  for  public  release; 
distribution  is  unlimited. 


Navy  Personnel  Research  and  Development  Center 
San  Diego,  California  92152-6800 


REPORT  DOCUMENTATION  PAGE 

Form  Approved 

OMBNo.  0704-0188 

Public  reporting  burden  for  thi*  collection  of  information  is  estimated  to  avenge  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  dau  sources,  gathering 
and  maintaining  the  dau  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  informa¬ 
tion,  including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204, 
Arlington,  VA  22202-4302,  and  to  the  Office  of  Management  and  Budget,  Paperwork  Reduction  Project  (0704-0188),  Washington,  DC  20503. 

1 .  AGENCY  USE  ONLY  (Leave  blank)  2.  REPORT  DATE 

October  1990 

3.  REPORT  TYPE  AND  DATES  COVERED 
Technical-Sep  88-Oct  89 

4.  TITLE  AND  SUBTITLE 

Long  Term  Memory  for  Knowledge  Taught  in  School 

5.  FUNDING  NUMBERS 

Program  Element  0602233N, 

Work  Unit  RM33T23.05 

6  AUTHOR(S) 

George  B.  Semb,  John  A.  Ellis,  and  William  E.  Montague 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Navy  Personnel  Research  and  Development  Center 

San  Diego,  California  92152-6800 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

NPRDC-TR-91-1 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

Chief  of  Naval  Research  (ONT-222) 

800  Nonh  Quincy  Street,  Arlington,  VA  22217-5000 

10.  SPONSORING/MONITORING 

AGENCY  REPORT  NUMBER 

11.  SUPPLEMENTARY  NOTES 

12a.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  is  unlimited. 

12b.  DISTRIBUTION  CODE 

13  ABSTRACT  (Maximum  200  words) 

This  study  examined  students’  retention  of  knowledge-based  college  course  content  after  4-  and  11 -months.  Retention  was 
evaluated  for  group-paced  and  self-paced  instructional  delivery.  The  study  focused  on  the  ability  to  recognize  information  learned  in 
school.  There  were  no  measures  of  the  ability  to  recall  information  or  solve  problems.  The  results  indicated  that;  (1)  students  scored 
approximately  85  percent  after  4  months  and  about  80  percent  after  1 1  months;  (2)  self-paced  students  performed  significantly  better 
on  all  tests  at  both  intervals  than  did  group-paced  students;  (3)  the  amount  of  information  lost  did  not  differ  significantly  between 
groups  across  the  two  retention  intervals  and;  (4)  the  amount  of  information  lost  relative  to  initial  learning  did  not  differ  significantly 
between  groups  across  the  two  retention  intervals. 

Because  the  multiple-choice  and  true-false  test  item  formats  used  in  this  study  are  identical  to  those  used  tn  most  Navy 
introductory  technical  courses  and  because  of  the  similarities  in  the  type  of  information  taught  in  the  college  course  and  Navy  courses 
(e.g.  names,  definitions,  principles),  it  can  be  concluded  that  students  will  be  able  to  recognize  knowledge  learned  in  Navy  technical 
classrooms  even  after  an  1 1 -month  retention  interval.  Further  research  is  planned  to  investigate  students’  ability  to  recall  information 
learned  in  school.  /  • 

14  SUBJECT  TERMS 

Long  term  memory,  self-paced  instruction,  retention ,  _  t 

'  / 

15  NUMBER  OF  PAGES 

18 

16.  PRICE  CODE 

17.  SECURITV CL*CS!FICA- 
TION  OF  REPORT 
UNCLASSIFIED 

18.  SECURITY’ CLASSIFICA¬ 
TION  OF  THIS  PAGE 
UNCLASSIFIED 

19.  SECURITY  CLASSIFICA¬ 
TION  OF  ABSTRACT 
UNCLASSIFIED 

20.  LIMITATION  OF  ABSTRACT 

UNLIMITED 

NSN  7540-01-280-5500 


Standard  Form  298  (Rev  2-89) 
Prescribed  by  ANSI  Sid  Z39- 18 
298-102 


FOREWORD 
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SUMMARY 


Problem  and  Background 

Military  trainees  frequently  do  not  retain  much  of  what  they  have  been  taught  in  school  by  the 
time  they  start  working  on  the  job.  However,  much  of  the  research  on  this  problem  has  focused  on 
procedural/psychomotor  skills  and  tasks  (e.g.,  performing  preventive  maintenance,  operating 
equipment).  While  the  research  on  skill  retention  has  been  extensive,  there  has  not  been  much 
work  on  memory  for  knowledge  taught  in  school. 

Objective 

The  primary  objective  of  this  effort  was  to  determine  the  amount  of  information  students  retain 
from  an  introductory  college  course  4-  and  11-months  after  completing  the  course.  The  course 
was  an  introductory  child  psychology  course  that  taught  facts,  concepts,  and  principles  and 
required  some  problem  solving. 

A  second  objective  was  to  determine  if  the  amount  of  information  retained  by  students  taught 
in  a  self-paced  course  differs  from  that  retained  by  students  taught  in  a  group-paced,  lecture 
discussion  course. 

Method 

The  experiment  compared  students  taught  by  two  different  methods,  self-paced  versus  group- 
paced  instruction,  on  original  learning,  retention  after  4  months  and  retention  after  1 1  months. 
Because  it  was  impossible  to  retest  all  the  students  who  took  the  4-month  retention  exam  at  the  11- 
month  interval,  the  two  retention  intervals  involved  different  numbers  of  students-the  11 -month 
group  was  a  subset  of  the  4-month  group.  In  the  group-paced  condition,  there  were  36  subjects  at 
the  4-month  interval  and  18  at  the  11 -month  interval.  In  the  self-paced  condition,  there  were  44 
subjects  at  the  4- month  interval  and  19  at  the  1 1 -month  interval.  The  primary  performance  measure 
was  the  score  on  the  end-of-course  exam  composed  of  multiple-choice  and  true-false  items  that 
tested  student  ability  to  recognize  information  learned  during  the  course.  There  were  four  forms  of 
the  exam  and,  at  the  4-month  interval,  students  were  retested  with  the  same  form  they  took  at  the 
end  of  the  course  and  were  also  tested  with  a  different  form.  Students  took  the  same  two  tests  at 
the  1 1 -month  interval  that  they  had  taken  at  the  4-month  interval. 

Results 

A  repeated  measures,  analysis  of  variance  (ANOVA),  revealed  significant  decline  in 
performance  between  the  end-of-course  test  and  the  4-month  retention  test,  and  between  the  4- 
month  and  11-month  interval.  Specifically,  the  results  were  that:  (1)  students  retained 
approximately  85  percent  after  4  months  and  about  80  percent  after  1 1  months;  (2)  self-paced 
students  performed  significantly  better  on  all  tests  at  both  intervals  than  did  group-paced  students; 
(3)  the  amount  of  information  lost  did  not  differ  significantly  between  groups  across  the  two 
retention  intervals;  and  (4)  the  amount  of  information  lost  relative  to  initial  learning  did  not  differ 
significantly  between  groups  across  the  two  retention  intervals. 

Concluding  Remarks 

It  is  important  to  emphasize  that  this  study  focused  on  the  ability  to  recognize  information 
learned  in  school.  There  were  no  measures  of  the  ability  to  recall  information  or  solve  problems. 
However,  the  types  of  recognition  test  items  used  in  this  study  (i.e.  multiple-choice  and  true-false 
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items)  are  identical  to  those  used  in  most  Navy  introductory  courses.  Therefore,  because  the  test 
item  formats  were  identical  and  because  of  the  similarities  in  the  type  of  information  taught  in 
University  of  Kansas  course  and  in  introductory  Navy  technical  courses  (e.g.  names,  definitions, 
principles),  it  can  be  concluded  that  students  wili  be  able  to  recognize  knowledge  learned  in  Navy 
technical  classrooms  even  after  an  11-month  retention  interval.  Further  research  is  planned  to 
investigate  students’  ability  to  recall  information  learned  in  school. 
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INTRODUCTION 


Problem  and  Background 

Military  trainees  frequently  do  not  retain  much  of  what  they  have  been  taught  in  school  by 
the  time  they  start  working  on  the  job  (Schendel,  Shields,  &  Katz,  1978;  Vineberg,  1975;  Hagman 
&  Rose,  1983).  Some  of  the  factors  that  have  been  shown  to  contribute  to  this  loss  include  type  and 
complexity  of  the  job-task,  amount  and  quality  of  the  initial  training,  length  of  time  without 
practice,  and  amount  and  quality  of  experience  encountered  on-the-job  (Montague,  Wetzel,  & 
Konoske,  1983;  Farr,  1986). 

Much  of  the  research  on  skill  and  knowledge  retention  has  focused  on  procedural/ 
psychomotor  skills  and  tasks  (e.g.,  performing  preventive  maintenance,  operating  equipment). 
After  a  number  of  years  of  research  on  these  types  of  tasks,  the  Army  integrated  the  findings  and 
published  a  User’s  Manual  for  Predicting  Military  Task  Retention  with  an  accompanying  form, 
TRADOC  Form  321  -R. 

While  the  research  on  skill  retention  has  been  extensive,  memory  for  knowledge  taught  in 
school  has  received  little  attention  (Neisser,  1982).  The  conventional  wisdom  is  that  most  of  what 
is  learned  in  school  is  quickly  forgotten.  In  a  book  on  memory,  Higbee  (1977)  states  that  “people 
. . .  forget  what  they  learned  in  school  (usually  within  a  short  time  after  an  exam).”  Bahrick  (1979) 
says  that  “much  of  the  information  acquired  in  classrooms  is  lost  soon  after  final  examinations  are 
taken.”  However,  studies  conducted  in  the  first  half  of  this  century  provide  some  evidence  that  this 
may  not  be  the  case  (Johnson,  1936;  Wert,  1937).  In  a  more  recent  study,  Ellis  (1980)  found  that 
Navy  ‘A’  School  students  lost  very  little  information  over  a  6-month  retention  period. 

Memory  for  knowledge  learned  in  school  is  a  very  important  issue.  In  the  civilian  world,  the 
very  existence  of  high  school  and  college  education  rests  on  the  assumption  that  people  remember 
something  of  what  they  learn.  In  the  military,  knowledge  retention  is  even  more  critical. 
Presumably,  the  knowledge  that  students  learn  in  military  classrooms  is  prerequisite  for  knowing 
when  and  how  to  perform  jobs  and  tasks  in  the  real  world.  Thus,  it  is  important  to  know  how  much 
and  what  kind  of  knowledge  students  remember.  Unfortunately,  according  to  Neisser  (1982),  “It  is 
difficult  to  find  even  a  single  study,  ancient  or  modem,  of  what  is  retained  from  academic 
instruction.”  Recently,  Bahrick  (1984)  found  that  even  after  50  years,  subjects  retained  a 
substantial  amount  of  Spanish  learned  in  school.  However,  not  much  research  has  studied  memory 
for  the  type  of  knowledge  that  college  students  learn  in  introductory  courses  or  Navy  personnel 
learn  in  apprentice  training  courses. 

Although,  the  content  of  college  introductory  courses  and  Navy  apprentice  training  courses 
may  be  quite  different,  the  strategies  for  teaching  and  learning  that  content  are  very  much  the  same. 
In  some  cases,  such  as  college  physics  and  Navy  basic  electricity  training,  even  the  content  is 
similar.  Basically,  both  types  of  courses  teach  facts  (e.g,  names,  dates,  definitions),  introductory 
concepts  and  principles,  and,  in  some  courses,  problem  solving  skills.  Thus,  examining  what 
college  students  retain  from  an  introductory  college  course  should  tell  us  something  about  what 
Navy  personnel  will  retain  from  a  Navy  apprentice  course. 
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Objective 


The  primary  objective  of  this  effort  was  to  determine  the  amount  of  information  students 
retain  from  an  introductory  college  course  4-  and  11-months  after  completing  the  course.  The 
course  was  an  introductory  child  psychology  course  that  taugh*  facts,  concepts,  and  principles  and 
required  some  problem  solving. 

A  second  objective  was  to  determine  if  the  amount  of  information  retained  by  students  taught 
in  a  self-paced  course  differs  from  that  retained  by  students  taught  in  a  group-paced,  lecture 
discussion  course. 


METHOD 


Subjects  and  Setting 

Subjects  were  students  enrolled  in  14  sections  of  a  one -semester  introductory  child 
psychology  course  at  the  University  of  Kansas.  The  course  was  taught  in  two  formats:  group- 
paced,  lecture  discussion,  two  sections  and  self-paced,  12  sections.  Only  students  who  were  not 
psychology  majors,  who  took  the  comprehensive  final  exam  at  the  end  of  the  semester,  and  who 
agreed  to  participate  in  the  study  by  signing  a  University-approved  research  consent  form  and  for 
whom  American  College  Test  (ACT)  test  scores  and  University  grade-point  averages  were 
available  were  considered  for  inclusion  as  subjects.  Psychology  majors  were  excluded  because 
they  were  likely  to  take  additional  psychology  courses  during  the  retention  interval.  Students  who 
were  actually  included  as  subjects  took  both  forms  of  the  retention  exams  at  each  of  the  two 
retention  intervals.  Because  all  the  students  who  took  the  4-month  retention  exam  could  not  be 
retested  at  the  11-month  interval,  the  two  retention  intervals  involved  different  numbers  of 
students-the  11-month  group  was  a  subset  of  the  4-month  group.  The  group-paced  condition 
included  36  subjects  at  the  4-month  interval  and  18  at  the  11-month  interval;  the  self-paced 
condition,  44  subjects  at  the  4-month  interval  and  19  at  the  1 1 -month  interval. 

Instructional  Procedures 

The  course  was  divided  into  twelve  content  units,  each  of  which  covered  about  one  chapter 
in  the  course  text  and  accompanying  study  guide.  All  students  took  a  14-item  quiz  at  the  end  of 
each  unit  and  a  28-item  midterm  exam  after  every  fourth  unit.  Quizzes  and  midterms  consisted  of 
a  mixture  of  true-false  and  multiple-choice  items.  The  course  ended  with  a  comprehensive  72-item 
final  exam. 

Self-paced  Instruction 

Students  in  the  self-paced  sections  worked  at  their  own  rate.  They  reported  to  class  to  work 
with  the  teaching  staff  and  to  take  tests.  Each  section  had  42  students  and  a  staff  of  five 
undergraduate  proctors  and  a  graduate  teaching  assistant. 

There  were  four  forms  of  the  unit  quizzes  and  midterm  exams.  Students  who  answered  less 
than  11  of  14  on  a  unit  quiz  correctly  were  required  to  (1)  show  their  proctor  a  completed  study 
guide  for  that  unit  and  (2)  retake  an  alternate  form  of  the  quiz.  To  progress  in  the  course,  students 
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had  to  retake  unit  quizzes  until  they  earned  a  score  of  1 1  or  better.  They  could  take  unit  quizzes  as 
many  as  four  times. 

Students  who  answered  less  than  14  of  the  28  items  on  the  midterm  exam  correctly  were 
required  to  take  an  alternate  form;  those  who  answered  more  than  14  correctly  could  retake  it  to  try 
to  improve  their  score.  The  highest  score  on  the  midterm  exam  counted  toward  the  student’s  grade 
in  the  course.  Students  could  only  take  the  midterms  twice. 

There  were  three  parallel  forms  of  the  comprehensive  end-of-course  exam.  Students  could 
take  the  exam  twice  and  the  highest  score  was  counted. 

Group-paced  Instruction 

Students  in  the  two  group-paced  sections  met  with  an  instructor  twice  a  week.  The  instructor 
lectured,  administered  quizzes  and  midterms,  and  discussed  course  content  with  students.  Group- 
paced  students  had  two  chances  to  take  each  quiz  (as  opposed  to  up  to  four  chances  for  self-paced 
students),  midterm,  and  the  final  and  the  highest  score  counted  toward  their  grade  in  the  course. 
The  final  exam  and  administration  procedures  for  the  group-paced  sections  was  the  same  as  for  the 
self-paced  sections. 

Experimental  Procedures 

The  primary  dependent  measure  in  this  study  was  the  comprehensive  final  exam.  Each  of 
the  four  parallel  forms  contained  16  true-false  items  and  56  multiple-choice  items.  Three  of  the 
forms  were  used  in  the  classroom  as  the  final  exam;  the  fourth  form  was  reserved  for  use  as  one  of 
the  retention  measures.  Each  form  contained  6  items  for  each  of  the  12  units  in  the  course.  The 
items  emphasized  the  major  concepts  covered  in  the  course  and  were  selected  from  a  pool  of  over 
1 ,300  items.  Within  each  item  type  (true-false  and  multiple-choice)  and  within  each  concept  area, 
four  questions  were  selected  for  each  form.  Items  were  assigned  to  forms  randomly  from  this 
stratified  sample.  No  item  appeared  on  more  than  one  form  of  the  exam.  Furthermore,  each  form 
contained  no  more  than  36  items  that  appeared  elsewhere  in  the  course  (quizzes,  review  exams,  and 
study  gu.de). 

Since  all  four  forms  of  the  exam  had  been  used  before  this  study,  we  were  able  to  analyze 
how  equivalent  they  were.  The  differences  among  forms  was  not  statistically  different.  The  final 
exams  were  not  used  as  pretests  in  the  present  study;  however,  an  analysis  of  results  from 
comparable  forms  of  the  exams  from  previous  semesters  yielded  mean  pretest  scores  of  44  to  48 
percent  correct. 

Students  were  instructed  to  answer  each  item  on  the  exam  and  to  indicate  how  confident  they 
were  that  they  had  selected  the  correct  answer.  The  confidence  rating  scale  was  the  following  five- 
point  Likert  scale: 

5-Absolutely  certain  my  answer  is  correct. 

4-Highly  certain  my  answer  is  correct. 

3— Moderately  certain  my  answer  is  correct. 

2-Only  slightly  certain  my  answer  is  correct. 

1 -Uncertain  (a  guess). 
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The  first  set  of  retention  exams  was  administered  4  months  after  the  course  ended.  Students 
who  met  the  eligibility  criteria  were  contacted  by  telephone  in  mid-March  and  invited  to  take  the 
retention  exams  later  in  the  month.  As  an  incentive  to  participate,  they  were  offered  a  chance  to 
participate  in  a  “lottery”  in  which  they  had  a  1  in  10  chance  to  win  $15  and  a  1  in  100  chance  to 
win  $100.  All  students  who  agreed  to  participate  took  two  retention  exams:  One  exam  they  had  not 
seen  before  (different)  and  the  identical  form  on  which  they  had  scored  highest  during  the  previous 
semester  (same).  The  order  of  presentation  of  the  exams  was  counter-balanced;  students  were 
assigned  to  same-different  exam  or  different- same  exam  orders  at  random. 

The  second  set  of  retention  exams  was  administered  11  months  after  the  course  ended. 
Students  were  contacted  by  telephone  in  mid-October  and  invited  to  take  the  retention  exams  later 
in  the  month.  As  an  incentive,  subjects  were  offered  $10  to  retake  the  exams.  Students  took  the 
same  two  retention  exams,  the  form  they  had  taken  at  the  4-month  retention  test  (different)  and  the 
identical  form  on  which  they  had  scored  the  highest  during  the  semester  they  took  the  course 
(same). 

The  order  of  presentation  of  same/different  exams  was  again  counter-balanced  for  same 
versus  different  exam  forms  and  for  4-  versus  1 1 -month  intervals.  That  is;  there  were  four  possible 
combinations:  (1)  4-month,  same-different;  11-month,  same-different;  (2)  4-month,  same- 
different;  11 -month,  different-same;  (3)  4-month,  different-same;  11 -month,  same-different;  and 
(4)  4-month,  different-same;  11 -month,  different-same.  Students  again  were  asked  to  rate  their 
confidence  in  the  correctness  of  their  responses. 

RESULTS  AND  DISCUSSION 

The  two  retention  intervals  involved  different  numbers  of  students-the  1 1 -month  group  was 
a  subset  of  the  4-month  group.  Data  for  each  group  were  analyzed  separately. 

Demographic  Variables 

The  group-paced  and  self-paced  subjects  did  not  differ  significantly  on  any  of  the 
demographic  measures  for  either  of  the  retention  intervals.  At  4  and  1 1  months,  there  were  no 
significant  differences  for  ACT-Composite,  for  grade-point  average,  or  for  year  in  school. 

Same/Different  Exam  Order  Effects 

A  repeated  measures,  analysis  of  variance  (ANOVA),  revealed  a  significantly  higher  score 
(percentage  correct)  for  self-paced  students  at  the  end  of  4  months,  F(l,76)  =  40.2,  p  <  0.01,  MSe 
=  55.6,  but  no  significant  differences  for  the  order  of  presentation  of  the  exams  (different-same 
versus  same-different).  Since  order  of  presentation  produced  no  effect,  it  was  not  part  of  the 
analyses  of  the  1 1 -month  group. 

Academic  Performance 

Overall  Percentage  Correct 

Table  1  presents  the  performance  means  for  the  4-month  retention  group  on  the  end-of-course 
and  both  retention  exams.  A  2  (sdf-paced  vs.  group-paced)  by  3  (end-of-course  score  vs.  same 
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score  vs.  different  score)  repeated  measures  ANOVA  revealed  significantly  higher  scores  for  self- 
paced  students,  F(l,78)  =  41.5,  p  <0.01,  MSe  =  84.6,  and  significantly  lower  scores  for  both  groups 
at  4-months  versus  end  of  the  course,  F(2,156)  =  215.0,  p  <  0.01,  MSe  =  8.9.  There  was  no 
significant  interaction.  Post-hoc  comparison  revealed  that  scores  on  the  (1)  same  form  of  the  exam 
were  higher  than  on  the  different  fo^n,  t(79)  =  7.5,  p  <  0.01,  (2)  end  of  course  exam  were  higher 
than  on  same  form  4  months  later,  ij9)  =  13.6,  p  <  0.01,  and  (3)  end  of  course  exam  were  also 
higher  than  on  the  different  form  4  months  later,  t(79)  =  21.4,  p  <  0.01. 

Table  1 

Performance  Means  for  the  4-Month  Interval 
(Percent  Correct  by  Condition) 


Self-pace 

Group-paced 

End  of  Course  Exam 

87.3 

76.1 

Same  Form 

78.6 

68.5 

Loss 

-8.7 

-7.6 

Relative  Loss 

-9.9 

-10.0 

Different  Form 

73.5 

62.8 

Loss 

-13.8 

-13.3 

Relative  Loss 

-15.8 

-17.5 

Table  2  presents  the  performance  means  for  the  11 -month  retention  group  on  the  end  of 
course  and  both  retention  exams.  A  2  (self-paced  vs.  group-paced)  by  3  (end-of-course  score  vs.  4- 
month  same  score  vs.  1 1 -month  same  score)  repeated  measures  ANOVA  for  the  same  form  of  the 
exam  revealed  significantly  higher  scores  for  self-paced  students,  F(l,35)  =  14.7,  p  <  0.01,  MSe  = 

86.3,  and  significantly  lower  scores  after  1 1  months  for  all  students  compared  to  end-of-course  and 
4-month  performance,  F(2,70)  =  72.2,  p  <  0.01,  MSe  =  10.5.  There  were  no  significant  interactions. 
An  identical  repeated  measures  ANOVA  for  different  forms  of  the  exam  revealed  similar  findings 
for  teaching  conditions,  F(  1 ,35)  =  24.9,  p  <  0.01,  MSe  =  46.4,  and  retention  intervals,  F(2,70)  = 

98.4,  p  <  0.01,  MSe  =  16.6.  Again,  there  were  no  significant  interactions.  All  post-hoc  t-tests 
comparing  the  various  forms  and  intervals  were  statistically  significant  at  p  <  0.01. 

Loss 

Tables  1  and  2  present  the  loss  in  score  between  the  end  of  the  course  and  the  two  retention 
exams.  A  2  (self-paced  vs.  group-paced)  by  2  (same  vs.  different)  ANOVA  showed  that,  at  the  end 
of  4  months,  the  only  significant  difference  was  that  scores  for  the  same  form  were  higher  than 
scores  for  the  different  form  of  the  exam,  F(l,78)  =  55.9,  p  <  0.01,  MSe  =  10.8.  There  were  no 
significant  differences  between  groups,  nor  were  there  any  significant  between-or-within  subject 
interactions.  A  2  (self-paced  vs.  group-paced)  by  2  (4  months  vs.  1 1  months)  by  2  (same  vs. 
different)  ANOVA  revealed  no  significant  effects  for  teaching  condition.  Scores  were  significantly 
lower  after  1 1  months,  F(  1 ,35)  =  25.4,  p  <  0.05,  MSe  =  25.1  and  scores  on  the  same  form  were 
higher  than  on  the  different  form  at  both  4  and  1 1  months,  F(  1 ,35)  =  1 3. 1 ,  p  <  0.01 ,  MSe  =  11.1. 
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Table  2 


Performance  Means  for  the  11 -Month  Interval 
(Percent  Correct  by  Condition) 


Self-paced 

Group-paced 

End  of  Course  Exam 

87.3 

78.4 

Same  Form 

76.3 

64.6 

Loss 

-11.0 

-13.8 

Relative  Loss 

-12.6 

-17.6 

Different  Form 

68.9 

61.0 

Loss 

-18.4 

-17.4 

Relative  Loss 

-21.1 

-22.2 

Relative  Loss 

Tables  1  and  2  present  the  score  lost  relative  to  initial  learning  for  the  4-month  and  1 1 -month 
retention  groups.  Relative  loss  was  defined  as  the  score  lost  adjusted  for  initial  learning  and  was 
calculated  by  dividing  the  percentage  lost  by  the  percentage  correct  store  at  the  end  course  (initial 
learning). 

A  2  (self-paced  vs.  group-paced)  by  2  (4  months  vs.  1 1  months)  by  2  (same  vs.  different) 
ANOVA  on  relative  loss  revealed  no  significant  effects  for  teaching  condition.  Scores  were 
significantly  lower  after  1 1  months,  F(l,35)  =  21.4,  p  <  0.05,  MSe  =  52.8  and  scores  on  the  same 
form  were  higher  than  on  the  different  format  both  4  and  11  months,  F(1.35)  =  18.7,  p  <0.01,  MSe 
=  71.2. 

Academic  Performance:  Discussion 

The  results  for  all  measures  of  academic  performance  support  the  position  that  students 
retained  a  great  deal  of  what  they  had  originally  learned:  approximately  85  percent  after  4  months 
and  about  80  percent  after  1 1  months.  These  finding  are  congruent  with  what  others  have  found 
(e.g.,  Ellis,  1980;  Bahrick,  1984)  in  studies  of  retention  among  adults  with  knowledge-based 
materials.  They  do  not  support  the  belief  that  most  of  what  is  learned  in  school  is  quickly  forgotten. 

These  results  also  clearly  support  others  who  have  found  that  the  amount  of  original  learning 
is  an  important  determinant  of  retention  (Farr,  1986).  This  is  supported  by  the  finding  that  the 
higher  level  of  performance  for  self-paced  students  on  the  end  of  course  exam  was  maintained 
across  the  4-month  and  1 1 -month  retention  intervals. 

Further,  in  this  study,  the  teaching  method  made  a  difference  in  both  the  amount  of  original 
learning  and  the  amount  retained.  The  performance  of  self-paced  students  was  superior  to  that  of 
group-paced  students  on  both  measures.  One  explanation  for  this  finding  is  that  the  groups  received 
different  amounts  of  testing  during  the  course.  Self-paced  students  could  take  each  quiz  up  to  four 
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times  while  group-paced  students  had  a  maximum  of  two  chances.  An  analysis  of  the  quiz  data 
shows  that  self-paced  students  averaged  16.2  quizzes  during  the  semester  while  group-paced 
students  averaged  1 1.3  (t(78)  =  5.37  p  <  .001).  In  terms  of  test  items,  this  means  that  self-paced 
students  were  exposed  to  227  items  while  group-paced  students  only  saw  158.  Self-paced  students 
were  also  required  to  complete  a  study  guide  and  consult  with  their  proctor  when  they  did  not  meet 
criterion  on  a  quiz.  Thus,  self-paced  students  had  more  exposure/practice  with  the  type  of  items 
that  were  on  the  final  exam  as  well  as  more  enriched  study  and  feedback  on  the  areas  in  which  they 
were  weak.  This  differential  treatment  could  account  for  the  observed  performance  differences. 

Finally,  although  there  were  significant  differences  between  the  self-paced  and  group-paced 
condition  in  the  amount  learned  and  retained,  neither  the  amount  of  information  lost  nor  the  amount 
lost  relative  to  initial  learning  differed  significantly  between  groups  across  the  two  retention 
intervals. 

Confidence  Ratings 

Overall  Correct  Confidence 

Table  3  presents  the  confidence  rating  data  for  the  4-month  group.  For  this  study,  we  defined 
correct  confidence  as  the  sum  of  all  the  confidence  ratings  for  items  answered  incorrectly 
subtracted  from  the  sum  of  the  ratings  for  items  answered  correctly.  This  means  that,  if  someone 
got  a  perfect  score  on  the  72  item  test  and  rated  each  answer  as  5  (absolutely  certain),  they  would 
receive  a  correct  confidence  score  of  360  (72  items  correct  times  5  minus  0  for  no  items  answered 
incorrectly).  If  a  person  missed  all  the  items  and  also  rated  each  answer  as  5,  the  correct  confidence 
score  would  be  -360  (0  items  scored  correctly  minus  5  times  72  items).  The  measure  was  designed 
to  combine  both  the  correctness  of  the  response  and  the  student’s  confidence  about  it. 

Table  3 

Confidence  Means  for  the  4-Month  Interval 


Self-paced 

Group-paced 

End  of  Course  Exam 

223.7 

151.7 

Same  Form 

167.5 

109.9 

Actual  Loss 

-56.2 

-41.8 

Percent  Loss 

-25.1 

-27.5 

Different  Form 

135.0 

79.8 

Actual  Loss 

-88.7 

-71.9 

Percent  Loss 

-39.6 

-47.4 

A  2  (self-paced  vs.  group-paced)  by  3  (end-of-course  score  vs.  same  score  vs.  different  score) 
repeated  measures  ANOVA  for  the  4-month  group  showed  that  self-paced  students  were 
significantly  more  confident  than  group-paced  students,  F(l,78)  =  39.8,  p  <  0.01,  MSe  =  565.8,  and 
that  confidence  declined  significantly  across  the  4-month  retention  interval,  F(2,156)  =  199.5,  p  < 
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0.01,  MSe  =  649.7.  There  was  no  significant  interaction.  Post- hoc  comparison  revealed  that 
confidence  was  greater  for  the  (1)  same  versus  different  form  of  the  exam,  t(79)  =  8.08,  p  <  0.01, 
(2)  end-of-course  versus  same  form,  t(79)  =  12.1,  p  <  0.01,  and  (3)  end-of-course  versus  different 
form,  t(79)  =  19.3,  p  <  0.01. 

Table  4  presents  correct  confidence  means  for  the  11 -month  retention  group.  An  2  by  3 
repeated  measures  ANOVA  for  the  same  form  of  the  exam  revealed  that  self-paced  students  were 
significantly  more  confident,  F(l,35)  =  1 1.43,  p  <0.01,  MSe=  6320.2,  and  that  confidence  declined 
across  the  4-  and  1 1-month  retention  intervals,  F(2,70)  =  91.5,  p  <  0.01,  MSe  =  1 182.2.  There  was 
no  significant  interaction.  An  identical  repeated  measures  ANOVA  for  different  forms  of  the  exam 
showed  that  self-paced  students  were  significantly  more  confident,  F(l,35)  =  17.2,  p  <  0.01,  MSe 
=  3414.5.3,  and  that  confidence  declined  significantly  across  th~  4-  and  11-month  retention 
intervals,  F(2,70)  =  154.0,  p  <  0.01,  MSe  =  1073.9.  Again,  there  wa  o  significant  interaction.  All 
post-hoc  t-tests  comparing  the  various  forms  and  intervals  were  statistically  significant  at  p  <  0.01 . 

Table  4 

Confidence  Means  for  the  11-Month  Interval 


Self-paced 

Group-paced 

End  of  Course 

226.8 

164.9 

Same  rorm 

109.7 

66.2 

Actual  Loss 

-117.1 

-98.7 

Percent  Loss 

-51.6 

-59.8 

Different  Form 

73.2 

52.8 

Actual  Loss 

-153.6 

-112.1 

Percent  Loss 

-67.7 

-67.9 

Actual  Loss 

Tables  3  and  4  present  the  loss  of  correct  confidence  at  the  end  of  each  retention  interval.  At 
the  end  of  4  months,  the  students  were  more  confident  of  their  responses  on  the  same  than  on  the 
different  form  of  the  retention  exams,  F(l,76)  =  62.3,  p  <  0.01,  MSe  =  622.2.  The  difference  in 
confidence  between  self-paced  and  group-paced  conditions  was  marginally  significant,  F(l,76)  = 
4.60,  p  <  0.05,  MSe  =  2118.2,  such  that  self-paced  lost  more  than  group-paced.  There  were  no 
significant  between-  or  within-subject  interactions.  At  the  end  of  11  months,  there  were  no 
significant  differences  between  teaching  conditions  for  the  same  form  of  the  retention  exam,  but 
self-paced  students  lost  marginally  more  on  the  different  form  than  group-paced  students,  F(l,36) 
=  4.69,  p  <  0.05,  MSe  =  3285.4. 
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Percent  Loss 


Tables  3  and  4  also  present  the  percent  of  correct  confidence  loss  relative  to  initial  confidence 
(at  end  of  course).  The  percent  loss  was  calculated  by  dividing  the  amount  of  confidence  lost  by 
the  amount  of  confidence  present  at  the  end  of  the  course  (initial  correct  confidence). 

At  the  end  of  4  months,  there  was  no  significant  difference  in  the  relative  amount  of 
confidence  lost  between  conditions  on  the  same  form  of  the  retention  exam;  however,  group-paced 
students  lost  marginally  more  confidence  than  self-paced  students  on  the  different  form,  F(l,79)  = 
5.54,  P  <  0.05,  MSe  =  330.5.  At  the  end  of  11  months,  there  were  no  significant  differences 
between  teaching  conditions  on  the  same  or  different  forms  of  the  exam. 

Confidence  Ratings:  Discussion 

Self-paced  students  were  significantly  more  correct  in  their  confidence  ratings  than  group- 
paced  students  at  the  end  of  the  course  and  at  the  end  of  both  of  the  retention  intervals.  Both  groups 
had  higher  confidence  ratings  on  the  same  form  of  the  retention  exam  than  on  the  different  form 
and  this  difference  was  constant  across  the  two  retention  intervals.  Further,  both  groups’ 
confidence  and  relative  confidence  decline  at  approximately  the  same  rate  for  both  forms  of  the 
retention  test  (there  was  a  marginal  difference  in  favor  of  the  self-paced  students  on  the  different 
form  at  the  4-month  interval).  The  explanation  for  the  differences  in  confidence  is  the  same  as  the 
explanation  for  the  differences  in  performance:  Self-paced  students  saw  more  test  items  and  had 
richer  more  extensive  feedback  than  group-paced  students.  Thus,  they  were  more  confident  of  their 
answers  on  the  final  exam. 

CONCLUDING  REMARKS  AND  FUTURE  EFFORTS 

In  the  introduction  in  this  study,  we  assumed  that  the  type  of  information  taught  in  this 
introductory  child  psychology  course  is  similar  to  the  type  of  information  taught  in  Navy  technical 
classrooms.  That  is,  the  University  of  Kansas  course  and  most  Navy  apprentice  classrooms 
emphasize  remembering  facts  (names,  functions,  definitions),  rules  (formulas,  laws),  and 
principles  (cause  and  effect  relationships).  Some  college  and  Navy  courses  deal  with  problem 
solving  (e.g.,  introductory  physics  in  college  and  basic  electricity  in  the  Navy)  and  many  Navy 
courses  (and  some  college  courses)  have  laboratory  sections  that  focus  on  learning  procedures. 
But,  the  primary  focus  in  both  types  of  course  is  on  remembering  information. 

A  second  similarity  between  this  study  and  Navy  courses  is  the  way  the  self-paced  students 
were  treated.  Granted  the  Navy  has  few  if  any  self-paced  courses,  but  the  goal  of  Navy  technical 
training  is  to  have  people  complete  training,  not  weed  them  out.  Thus,  the  opportunity  for  students 
to  remediate  and  retest  in  the  self-paced  course  is  similar  to  the  way  Navy  students  are  typically 
treated.  In  Navy  courses,  students  who  are  having  problems  are  assigned  to  night  study,  given  on- 
the-spot  remediation,  or  can  be  “set  back”  to  restudy/releam  the  course  material. 

These  similarities  mean  that  we  can  expect  retention  for  both  types  of  courses  to  be  similar. 
In  fact,  the  findings  in  this  study  are  very  similar  to  what  Ellis  (1980)  found  in  a  study  of  what  Navy 
students  remembered  from  an  introductory  propulsion  engineering  course  after  1  month  and  6 
months.  However,  there  are  two  concerns  that  affect  this  conclusion.  First,  all  the  test  items  in  the 
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Kansas  course  were  multiple -choice,  true-false,  or  matching.  That  is,  only  the  ability  to  recognize 
information  was  tested.  We  do  not  know  if  the  retention  of  test  items  that  required  recall  of 
information  would  be  similar;  however,  previous  research  on  recall  memory  has  found  lower 
performance.  We  are  currently  conducting  a  follow-up  study  to  investigate  this  issue.  The  follow¬ 
up  study  should  be  completed  in  FY  91.  The  second  concern  is  that  there  is  some  evidence  that 
students  come  into  the  child  psychology  course  with  some  prior  knowledge  of  the  content.  This  is 
less  likely  to  be  true  of  a  Navy  technical  course.  Therefore,  while  the  retention  may  still  be  similar, 
the  amount  of  learning  may  differ.  A  solution  to  this  concern  is  to  collect  data  on  a  Navy 
introductory  course  to  verify  and  validate  the  present  findings.  This  is  difficult  to  do  because  Navy 
students  are  hard  to  track  after  they  have  graduated  from  school.  Further,  it  is  difficult  to  determine/ 
control  what  Navy  students  experience  during  the  retention  interval.  Ideally,  their  work  should  be 
related  to  what  they  learned  in  school  which  would  confound  any  conclusions  about  amount  of 
information  remembered.  College  students,  on  the  other  hand,  are  likely  to  return  the  following 
year  (unless  they  graduate,  transfer,  or  drop  out)  and  it  is  easy  to  determine  what  courses  they  have 
taken  in  the  interim.  However,  the  possibility  will  be  explored. 
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